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Abstract 

The aim of this paper is to prove theorems for the Slepian-Wolf source coding and the broadcast channel 
coding (independent messages and no common message) based on the the notion of a stronger version of the hash 
property for an ensemble of functions. Since an ensemble of sparse matrices (with logarithmic column degree) 
has a strong hash property, codes using sparse matrices can realize the achievable rate region. Furthermore, 
extensions to the multiple source coding and multiple output broadcast channel coding are investigated. 
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I. Introduction 

l>' 

C\| ' The aim of this paper is to prove theorems for the Slepian-Wolf source coding (Fig. 1) introduced in [26] 

version of the hash property for an ensemble of functions introduced in [17] [18]. This notion provides a 

O 

sufficient condition for the achievability of coding theorems. Since an ensemble of sparse matrices also has 
a strong hash property, we can construct codes by using sparse matrices where the rate pair of the code is 
close to the boundary of the achievable rate region. When constructing codes, we employ minimum-divergence 
encoding and maximum-likelihood decoding. 

The achievable rate region for Slepian-Wolf source coding is derived in [26]. The technique of random bin 
coding is developed in [4] to prove the Slepian-Wolf source coding theorem. The achievability of the code 
using a pair of matrices is studied in [5]. The constructions of encoders using sparse matrices is studied in 
[20][27] and the achievability is proved in [16] by using maximum-likelihood (ML) decoding. In this paper, 
we construct codes based on the strong hash property, which unifies the results of [4] [5] [16]. 

To construct a broadcast channel code, we assume that independent messages are decoded by their respective 
receivers with small error probability. It should be noted that we assume neither "degraded" nor "less noisy" 
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and the broadcast channel coding (Fig. 2) introduced in [3]. The proof of theorems is based on a stronger 
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Fig. 1. Slepian-Wolf Source Coding 



Sender Receivers 



M — ► 






^YZ\X 


— <-Y — 




— -M 














— - Z — 


^z 1 


— W 



Fig. 2. Broadcast Channel Coding 



conditions on this channel. The capacity region for this channel is known only for some classes. An inner region 
on the two receiver general broadcast channel without a common message is derived in [15] and a simpler way 
of constructing codes is presented in [7]. The cardinality bound is investigated in [10] [11]. Outer regions are 
derived in [13] [21] [22]. Applications of sparse matrices (LDPC codes) to broadcast channels are investigated 
in [2] [23] [24] [25]. In this paper, we construct codes based on a strong hash property and we show that the rate 
pair of the constructed code is close to the inner bound derived in [7] [15]. 

The proof of all theorems is based on the notion of a hash property, which is a stronger version of that 
introduced in [17]. It is the extension of the ensemble of the random bin coding [4], the ensembles of linear 
matrices [5], the universal class of hash functions [6], and the ensemble of sparse matrices [16]. Two lemmas 
called 'collision-resistace property 1 ' (if the number of bins is greater than the number of items then there is an 
assignment such that every bin contains at most one item) and 'saturation property 1 ' (if the number of items is 
greater than the number of bins then there is an assignment such that every bin contains at least one item), which 
are proved in [17] and reviewed in Section III, are extended from a single domain to multiple domains. The 
extended collision-resistance property is used to analyze the decoding error of the Slepian-Wolf source coding 
and the extended saturation property is used to analyze the encoding error of the broadcast channel coding. It 
should be noted that the linearity of functions is not necessary for a strong hash property but it is expected that 
the space and time complexity of the code can be reduced compared with conventional constructions by using 
sparse matrices. This is a potential advantage of our approach. 

II. Definitions and Notations 
Throughout this paper, we use the following definitions and notations. 

The cardinality of a set U is denoted by \U\, U c denotes the complement of U, and U \ V = U n V c denotes 
the set difference. 

'in [17], they were called 'collision-resistant property' and 'saturating property,' respectively. We changed these terms following the 
suggestion of Prof. T.S. Han. 
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Column vectors and sequences are denoted in boldface. Let Au denote a value taken by a function A:U n — >• 
liaA at u = (ui, . . . , tt„) e U n , where W n and ImA = {Au : u e U n } are the domain and the image of the 
function, respectively. It should be noted that A may be nonlinear. When A is a linear function expressed by 
anlxn matrix, we assume that U = GF(q) is a finite field. For a set A of functions, let Im*4 be defined as 

ImA = (J ImA. 

AeA 

We define a set Ca{o) as 

Ca(o.) = {u : Au = a}. 

In the context of linear codes, Ca (a) is called a coset determined by a. The random variables of a function A 
and a vector a £ ImA are denoted by sans serif letters A and a, respectively. On the other hand, the random 
variables of a n-dimensional vector u is denoted by bold Roman letter U. 

Let p and p' be probability distributions and let q and q 1 be conditional probability distributions. Then entropy 
H(p), conditional entropy H(q\p), divergence D(p\\p'), and conditional divergence D(q\\q'\p) are defined as 



H{q\p) = Y] q{u\v)p(v) log / 

q(u\v) 



d( p \\p')^J2p( u ) 1 °s$ 



p'(u) 



D(q || q'\p) = ^2p(v) q(u\v) log — 



q(u\v) 



q'{u\v) 

where we assume that the base of the logarithm is 2. 

Let [i\jy be the joint probability distribution of random variables U and V. Let fijj and fly be the respective 
marginal distributions and Hu\y be the conditional probability distribution. Then the entropy H(U), the 
conditional entropy H(U\V), and the mutual information I(U; V) of random variables are defined as 

H(U) = HifjLu) 

H(U\V) = Hfauw^v) 

I(U;V) = H(U) - H(U\V). 

A set of typical sequences Tjj a and a set of conditionally typical sequences Tu\v^{ v ) are defined as 

7c/, 7 = {u : D(v u \\fj,u) < 7} 
Tu\ v , 7 (v) = {u : D(v ulv \\n ulv \v v ) < 7} , 

respectively, where v u and v u \ v are defined as 

/ \ |{1 < i < n : Ui = u}\ 
v u {u) = 



v u \ v {u\v) 



n 

v uv (u,v) 



v v {v) 

For TcWxV and v e V, Tu and %t\v{ u ) are defined as 

Tu = {u : (u, v) e T for some t> e V"} 
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T u \v(v) = {u : (u,v) e T). 



We define %(•) as 



X(a = b) = 



X (a ± b) 



1, if a = b 

0, if a ^ b 

1, ifa^fe 
0, if a = b. 



Finally, for 7, 7' > 0, we define 

\U\\og(n + l) 
*u = 



Qa{i) =7 - V 2 7log 



^7 



\U\ 



Cu\vh'h) = i - V^yiog 



WW\ 



27 log \K\ 



27log^+ N1 ° g(n+1) 



|W| 



^7' log 



/2y 



27 log |W| 



|W||V|Iog(n + l) 



(1) 
(2) 

(3) 

(4) 

(5) 



|W||V| ■ 

where the product set U x V is denoted by UV when it appears in the subscript of these functions. These 
definitions will be used in the proof of theorems. 

III. Strong (a,/3)-HASH property 

A. Formal Definition and Basic Properties 

In the following, we introduce the strong hash property for an ensemble of functions. It requires a stronger 
condition than that introduced in [17]. It should be noted that the linearity of functions is not assumed in this 
section. 

Definition 1: Let A. = {A n }%Li be a sequence of sets such that A n is a set of functions A n : U n — > Inx4„. 
For a probability distribution p An on A n corresponding to a random variable A„ e A n , we call a sequence 
(A,p A ) = {(A n ,PA n )}%Li an ensemble. Then, (A,p A ) has a strong (oa, fi A )-hash property if there are two 
sequences a A = {a A (n)}^ =1 and (3 A = {P A (n)}^ =1 depending only on {pA n }n=o such that 



lim a A {n) = 1 

n— t- 00 

lim /3 A (n) = 



(HI) 
(H2) 



and 



^ p A „ ({A : Am = Am'}) < /3 A (n) (H3) 

ti'eu"\{u} 

p A „({A:A U =A U '})>^iL 

for any it e W™. Throughout this paper, we omit the dependence on n of A, A, A, a A , and (3 A . 

Let us remark on the conditions (H1)-(H3). These conditions require that the sum of the all collision 
probability which is far grater than l/|Ira4| vanishes as the block length goes to infinity. 
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Remark 1: The condition 



lim - log - — — = 
n^oo n |IrrL4„| 



is required for an ensemble in [17, Def. 1]. We omit this condition because it is unnecessary for the results 
presented in this paper. 

It should be noted that (^4, p A ) has a strong (1, 0)-hash property when A is a universal class of hash functions 
[6] and p A is the uniform distribution on A. The random bin coding [4] and the set of all linear functions [5] 
are examples of a universal class of hash functions. The strong hash property of an ensemble of sparse matrices 
is discussed in Section III-B. 

From the following lemma, we have the fact that the above definition of hash property is stronger than that 
introduced in [17]. It is proved in Appendix B. 

Lemma 1: If (^4, p A ) has a strong (qa, /3 A )-hash property, then 

£ pa ({A : Au = Art'}) < \TnT'\ + '^l^ + min{|T|, |T'|}/3 A (6) 
u'eT' 

for any T, T' C U n , that is, it has a (c* A , /3 A )-hash property introduced in [17, Definition 1], 

In the following, we review two lemmas of the hash property. It should be noted that Lemmas 2 and 3 are 
related to the collision-resistace property and the saturation property, respectively. These relations are explained 
in [17, Section III]. Let A be a set of functions A : U n — > 1mA, p a be the uniform distribution on 1mA, where 
random variables A and a be mutually independent, that is, 



p a (a) = < 



0, if a ^ Im.4 

PAa(A,a) =p A (A)p a (a) 

for any A and a. 

Lemma 2 ([17, Lemma 1]): If (A,pa) satisfies (6), then 

PA {{A : [Q \ {«}] n C A {Au) £ 0}) < jg^j + /3 A . 

for all Q G lA a and u G U n . 

Lemma 3 ([17, Lemma 2]): If (A, pa) satisfies (6) then 

PA. ({(A, a) : T n C A (a) = 0}) < a A 1 + M ^ + 1] 

for all T C U n . 

In the following, we consider the combination of two ensembles, where functions have the same domain. It 
should be noted that the assumption of a strong hash property makes it unnecessary to assume the linearity of 
a function for the hash property of the concatenated ensemble while the linearity of a function is assumed in 
[ 17] [1 8] [ 19] . The proof is given in Appendix C. 

Lemma 4: Assume that an ensemble (A.,p A ) of functions A : U n — > ImA satisfies an strong (a A , /3 A )-hash 
property and an ensemble (A!,p A >) of functions A' : U n — > Im.4.' satisfies an strong (ct/v, /3 A /)-hash property. 
Let p^ be the joint distribution on A = A x A' defined as 

Pa (A,A')=pa(A) Pa ,(A') 
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Then the ensemble (A.,p^) defined as 



for each A e A and A' e A'. Let (a^, f3~) be defined as 

a ^ = a a « A' 

A : U n — > Im.4 x Im.4' 
Ait = (Am, A'm) 

for each A = (A, A') e A and « gW" has a strong (a^, /3^)-hash property. 

In the following, we consider the combination of two ensembles, where the domains of functions are different. 
The following lemmas are essential for the proof of coding theorems presented in this paper. For a set A of 
functions A : U n — » Im.4, a set B of functions B : V" — > Im£>, let p a and pb the uniform distributions on 
Im.4 and InxB, respectively, where random variables {A, B, a, b} are mutually independent, that is, 

PABab(A, B, a, b) = p A (A)p B (B)p a (a)p b (b) 

for any A, B, a, and b. 

Lemma 5: If (A,pa) and (B,pe) satisfy (H3), then 

p AB ({(A, B):[Q\ {(u, w)}] n [Ca(A«) x C B {Bv)\ ± 0}) 



< \G\uaUb 



max |S W |v(w)| 


1 


max |</ V |w(«)l 


qb[/3a + 1] 


|InU| |IroB 





|Im.4||ImS| 
for all G C W x V" and («, u) e W" x V™. 

Lemma 6: If (.4,j?a) and (B,pb) satisfy (H3), then 

PABsb({(A,B,a,b) :Tn[C A {a) x C B (b)} = 0}) 

llmB 



+ /?a + /?b + /3a/?b (7) 



< Qa«b — 1 + ■ 



max |7w|v(«)l 



+ 



\T\ 

\lmA\\lmB\[/3 A + l3 B + l3 A /3 B + l] 
ITI 



a A [/3 B + l] |Im.A| 
+ 



max \T v \ u (u)\ 



a B [/3 A + l] 



ITI 



(8) 



for all TcW"xV" 

It should be noted that Lemmas 5 and 6 are related to the collision-resistance property (Lemma 2) and the 
saturation property (Lemma 3), respectively. Proof are given in Appendix D. 



B. Hash Property for Ensembles of Matrices 

In the following, we discuss the hash property for an ensemble of matrices. 

It has been discussed in the last section that the uniform distribution on the set of all linear functions has 
a strong (l,0)-hash property because it is a universal class of hash functions. In the following, we introduce 
another ensemble of matrices. 

First, we introduce the average spectrum of an ensemble of matrices given in [1]. Let U be a finite field and 
A be a set of linear functions A : U n — > U 1a . It should be noted that A can be represented by a I a x n matrix. 
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Let t(u) be the type 2 of u € W™, which is characterized by the empirical probability distribution v u of the 
sequence u. Let % be a set of all types of length n except t(0), where is the zero vector. For a probability 
distribution p& on a set of ^ x n matrices and a type t, let S(p A ,t) be defined as 

S(p A ,t) = ^ Pa(A)|{m e W n : Att = 0,t(«) = t}|, 

which is called the expected number of codewords that have type t in the context of linear codes. For given 
Ha C H, we define 0!a(ti) and (3 A (n) as 

|W| S(p A ,t) 
aA ( n ) = "177177" ' m ^ x ^7 77 ^ 

p A (n)= J2 5 (PA,<), (10) 
tew\7? A 

where u_4 denotes the uniform distribution on the set of all l^x n matrices. 
We have the following theorem. The proof is given in Section VIII-A. 

Theorem 1: Let (A., p A ) be an ensemble of matrices and assume that p A ({A : Au = 0}) depends on u only 
through the type t(u). If (a A ,f3 A ), defined by (9) and (10), satisfies (HI) and (H2), then (A.,p A ) has a strong 
(ot A , /3 A )-hash property. 

Next, we introduce the ensemble of q-ary sparse matrices introduced in [17], which is the q-ary extension 
of the ensemble proposed in [14]. Let U = G¥(q) and Ij, = nR for given < R < 1. We generate an ^ x n 
matrix A with the following procedure, where at most r random nonzero elements are introduced in every row. 

1) Start from an all-zero matrix. 

2) For each i S {1, . . . , n}, repeat the following procedure r times: 

a) Choose (j, a) e {1, . . . , l^} x [GF(<7) \ {0}] uniformly at random. 

b) Add 3 a to the (j, i)-element of A. 

Assume that r = O(logn) is even and let (A.,p A ) be an ensemble corresponding to the above procedure. Let 
"Ha C % be a set of types satisfying the requirement that the weight (the number of occurrences of non-zero 
elements) is large enough. Let (a A ,/3 A ) be defined by (9) and (10). Then a A measures the difference between 
the ensemble (A., p A ) and the ensemble of all x n matrices with respect to the high-weight part of the 
average spectrum, and (i A provides the upper bound of the probability that the code {u € U n : Au = 0} 
has low-weight codewords. It is proved in [17, Theorem 2] that (a A ,f3 A ) satisfy (HI) and (H2) by adopting 
an appropriate "Ha- Then, from Theorem 1, we have the fact that this ensemble has a strong (qa, /3 A )-hash 
property. It should be noted that the convergence speed of (a A ,f3 A ) depends on how fast r grows in relation 
to the block length. The analysis of (a A} (3 A ) is given in the proof of [17, Theorem 2]. 

2 In [17], it is called a histogram which is characterized by the number nv u of occurrences of each symbol in the sequence u. The type 
and the histogram is essentially the same. 

3 It should be noted that (j, i)-element of matrix is not overwritten by a when the same j is chosen again. 
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Fig. 3. Construction of Slepian-Wolf Source Code 



IV. Slepian-Wolf Source Coding 

In this section, we consider the Slepian-Wolf source coding illustrated in Fig. 1. The achievable rate region 
for this problem is given by 

Rx > H(X\Y) 
R Y > H(Y\X) 
Rx+Ry>H(X,Y), 

where (Rx,Ry) denotes an encoding rate pair. 

The achievability of the Slepian-Wolf source coding is proved in [4] and [5] for an ensemble of bin-coding and 
all g-ary linear matrices, respectively. The construction of encoders using sparse matrices is studied in [20] [27] 
and the achievability is proved in [16] by using ML decoding. The aim of this section is to demonstrate the 
proof of the coding theorem based on the hash property. The proof is given in Section VIII-B. 

We fix functions 

A : X n -> ImA 
B-.y n -> ImB, 

which are available for constructing encoders and a decoder. We define the encoders and the decoder (illustrated 
in Fig. 3) 

ipx ■ X n -> Im.4 
<Py ■■ y n -> ImB 
ip- 1 : ImA x ImB -> X n x y n 

as 

fx{x) = Ax 
<Py(v) = By 
^ _1 (a,6) = g A B{a,b), 
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where 

g AB (a,b) = arg min D{v x , y , \\hxy)- 

(x',y')eC A (a)xC B (b) 

It should be noted that the construction is analogous to the coset encoding/decoding in the case when A and 
B are linear functions. 

The encoding rate pair (Rx,Ry) is given by 

_ log \lmA\ 

Kx = 

n 

log |Im£>| 



Ry 



n 

and the error probability Errors y{A, B) is given by 

Ettot X y(A,B) = fi X Y ({(x,y) : <p _1 (vJx(a;), 7^ • 

We have the following theorem. It should be noted that X and y are allowed to be non-binary and the correlation 
of the two sources is allowed to be asymmetric. 

Theorem 2: Assume that (A.,p A ) and (&,p B ) have a strong hash property. Let (X, Y) be a pair of stationary 
memoryless sources. If (Rx,Ry) satisfies 

Rx>H(X\Y) (11) 

R Y > H{Y\X) (12) 

Rx+Ry >H(X,Y), (13) 

then for any S > and all sufficiently large n there are functions (sparse matrices) Aei and B e B such 
that 

Error X y(A,B) < S. 

Remark 2: Instead of a maximum-likelihood decoder, we use a minimum-divergence decoder, which is 
compatible with typical-set decoding. We can replace the minimum-divergence decoder by 

9AB( a , b ) = ar S , , max HxY{x',y'), (14) 

(x',y')GC A (a)xC B (b) 

x'eTx,-, 
y'eTv,-, 

where 7 > should be defined properly. The reason for introducing the conditions x' € 7x l7 and y' e Ty n 
is given at the end of Section VIII-B. 

V. Broadcast Channel Coding 

In this section we consider the broadcast channel coding problem illustrated in Fig. 2. A broadcast channel is 
characterized by the conditional probability distribution hyz\x, where X is a random variable corresponding to 
the channel input of a sender, and (Y, Z) is a pair of random variables corresponding to the channel outputs of 
respective receivers. In the following, we consider the case when a sender transmits two independent messages 
with nothing in common to two receivers. It is known that if a rate pair (Ry, Rz) satisfies 

Ry<I(U;Y) (15) 
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Fig. 4. Construction of Broadcast Channel Code 



Rz<I(V;Z) (16) 

Ry + Rz < I(U; Y) + I(V; Z) - I(U; V) (17) 

for some joint probability distribution nuvx on U x V x X, then there is a code for this channel such that 
the decoding error probability goes to zero as the block length goes to infinity, where the joint distribution of 
random variable (U, V, X, Y, Z) is given by 

HuvxYz(u,v,x,y,z) = Hrz\x(y, z\x)fi uvx (u, v, x). (18) 

This type of achievable region is derived in [15] and a simpler proof is given in [7]. Furthermore, it is shown 
in [10, Theorem 1] that it is sufficient to consider a joint probability distribution fijjvx satisfying \U\ < \X\, 
\V\ < \X\, and H(X\UV) = 0. In the following, we assume that \U\ < \X\, \V\ < \X\, and there is a function 
/ : U x V -> X such that 

fiuvx(u,v,x) = (i uv (u,v)x(x = f(u,v)), (19) 

where (19) is equivalent to H(X\UV) = 0. 

Let (Ry, Rz) be a pair of encoding rates. In the following, we assume that (Ry, Rz), (ty^z), and e satisfy 

r Y > H(U\Y) (20) 

r z > H(V\Z) (21) 

r Y + R Y < H(U) - e (22) 

r z + R z < H(V) - e (23) 
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r Y + R Y + r z + Rz < H{U, V) (24) 
r Y + R Y + r z + Rz > H{U, V) - e (25) 
for given fiyz\x, Huv, an d /■ It should be noted that there are (ry,rz) and e when (Ry,Rz) satisfies 



Ry < I(U;Y) 
Rz<I(V;Z) 
R Y +Rz< I(U; Y) + I(V; Z) - I(U; V). 
This fact is shown from the relations 

H(U)-H(U\Y) = I(U;Y) 
H(V)-H{V\Z)=I(V-Z) 
H(U, V) - H(U\Y) - H(V\Z) = I(U; Y) + I(V; Z) 

-I(U;V). 



We fix functions 



and vectors 



A : W -> ImA 
A' : U n -> ImA 1 

B : V n -> ImB 
B' : V" -> ImS' 

a e Iitl4 
be ImB 



available for an encoder and decoders satisfying 



r Y = 
Ry = 

rz = 
Rz = 



log |IraA| 

n 

log |Ira4'| 
n 

log | ImB | 

n 

log | ImB' | 



We define the encoder and the decoders 



as 



if : ImA' x ImB' -S- X n 
tfiy 1 : y n -> ImA' 
ip- 1 : Z n -> ImB' 

<p(m, i») = f{gAA<BB< (a, m, b, w j) 



(26) 
(27) 
(28) 
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ip Y 1 (y) = A'g A (a\y) 
<p-\z) = B'g B (b\z), 

where 

f(u,v) = (/(tti,Ui), . . .,f(u n ,v n )) 
gAA'BB'{a,m,b,w) = arg min D(v u > v >\\fj, uv ) 

(u',v'): 
u eC A {a)C\C A i(m) 

v'ec B (b)nc B ,(w) 
9A{a\y) = arg max /i[,|y(u'|y) 

u'eCA(a) 

g B {b\z) = arg max fj, v , z (v'\z). 
v'ec B (b) 

Let m and w be random variables corresponding to messages m and w, respectively, where the probability 
distributions p m and p w are given by 



Pm(m) 



Pw(w) 



ns^[ ifmeImi ' (29) 

0, if m ^ IroA' 
rj^gn if we ImB' 

lime i (3()) 
0, if to ^ ImB' 
and the joint distribution p m wYZ of the messages and outputs is given by 

p mw Yz(m,w,y,z) = p Y z\x(y, z\<p(rn,w))p M {rn)pw{w)- 

Let us remark an intuitive interpretation of the code construction, which is illustrated in Fig. 4. Assume that 
a and b are shared by the encoder and the decoder. For a, b, and messages m, w, the function qaa 1 b b 1 
generates a pair of typical sequence (it, v) G Tuv,i and it is converted to a channel input x by using a function 
/, where (u, v,x) is jointly-typical. One decoder reproduces u by using g A from a and a channel output y 
and another decoder reproduces v by using g B from b and a channel output z. Since A'u = m, B'v = w, 
and (u, v, x, y) is jointly typical, the decoding succeeds if the amount of information of a (resp. b) is greater 
than H(U\Y) (resp. H(V\Z)) to satisfy the collision-resistance property. On the other hand, the rate of a, m, 
b, and w should satisfy the saturation property to find a jointly typical sequences (u, v) with probability close 
to 1, that is, the right hand side of (8) should tend to zero as the block length goes to infinity. Then we can 
set the encoding rate of m and w satisfying (26)-(28). 

Let Error Y z\x(A, A' , B, B', a, b) be the decoding error probability given by 

Evvovy z{x (A, A', B,B', a, b) 

= 1- ^YZ\x(y,z\ip(m,w))pM(m)p w (w)x((p Y 1 (y) = m) X ((p z 1 (z) = w). (31) 

m .w .y ,z 

Then we have the following theorem. It should be noted that the channel is allowed to be asymmetric and 
non-degraded. 

Theorem 3: Let /j,yz\x be the conditional probability distribution of a stationary memoryless channel and 
[i-uvxYZ be defined by (18) and (19) for a given joint probability distribution \i\jy and a function /. For a 
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given (Ry,Rz), {ry,rz), and e satisfying (20)-(25), assume that ensembles (A.,p A ), (A.',p A >), (0,p B ), and 
(B',p B ,) have a strong hash property. Then, for any <5 > and all sufficiently large n, there are functions 
(sparse matrices) A E A, A' E A', B E B, B' E B' and vectors a E hnA, b E ImB such that 

Ratey > R Y -5 (32) 
Rate z > R z -S (33) 
Ettot yz \x(A, A', B, B' , a,b)<5 (34) 

By assuming that 8 — > 0, the rate of the proposed code is close to the boundary of the region specified by 
(15)— (17) for a given \ijjy and /. ■ 
Remark 3: It should be noted that the maximum-likelihood decoders qa and gs can be replaced by the 
minimum-divergence decoders g~A and gg defined as 

g A {a\y) = arg min D(v u , ly \\(j, ulY \v y ) 

u'£Ca{o.) 

g B (b\z) = arg min D(u v , lz \\fj, vlz \u z ), 
v'ec B (b) 

respectively. 

Remark 4: In [7] [15], the set of strong typical sequences are used for the construction of a broadcast channel 
code, where there is no structure of the set of codewords. It should be noted that the encoder and the decoder 
have to share the large (exponentially in the block length) table which indicates the correspondence between 
an index and codewords. The time complexity of the decoding grows exponentially in the block length. They 
represent obstacles to the implementation. In our construction, almost all codewords have strong typicality but 
the set of codewords have a structure specified by functions A, A', B, B 1 and vectors a, b. When the functions 
are linear, it is expected that the space and time complexity can be reduced compared with conventional 
constructions. 

Remark 5: It should be noted that the inner bound presented in the beginning of this section is not convex 
in general and we can apply the time sharing principle to obtain the convex hull of this region. It should also 
be noted that, according to [8, Page 9-36], there is a much larger inner region given by the set of a rate pair 
(Ry,Rz) satisfying 

Ry < I(S, U; Y) 

Rz < I(S,V;Z) 
Ry + Rz< I(S, U; Y) + I(V; Z\S) - I(U; V\S) 
Ry + Rz< I(U; Y\S) + I(S, V; Z) - I(U; V\S), 

for some probability distribution (isuv on 5 x tf x V and a function f: SxUxV^X. We presented the 
code construction for the smaller inner region, because it is a typical application of the extended saturation 
property (Lemma 6). It is a future challenge to obtain a code construction for the larger inner region based on 
the strong hash property. 
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VI. Construction of Minimum-Divergence Operation 
In this section, we introduce the construction of the minimum-divergence operation 

= ar S min D(i> u , v ,\\fj,uv), 

( U ',v')eC A (a)xC s (b) 

which is used to construct the encoder of the broadcast channel and the decoder of the Slepian-Wolf codes. 

The construction of this operation is introduced in [18] for a binary alphabet. However, we have to construct 
this operation for an alphabet that is a product space UxV. 

In the following, we assume that {0, 1} is a finite field, U = V = {0, 1} and that A and B are matrices on 
{0, 1}. It should be noted that our construction of broadcast channel codes can approach the boundary of the 
achievable region by assuming \X\ = 2 (see [10, Theorem 1]). 

Let t uv (u, v) e {0, . . . , n} be the number of occurrences of (u, v) <G {0, l} 2 in (it, v) e {0, 1}™ x {0, 1}™. 
For example, when n = 8 and 

u = 01001010 
v = 00101001, 

we have 

t uv (0,0) = 3 
t ttW (0,l) = 2 
t ttW (l,0) = 2 
t uv (l,l) = 1. 

Let t uv = {t uv (0,0),t uv (0, l),t uv (l,0),t uv (l,l)}. We have 

g^g(a,b) = arg min D(u UtVt \\fj, uv ), 

(ut,Vt): 

ten 

where 

(u, v) if 3(u, v) s.t. t uv = t, 



(u t ,v t ) = < 



Au = a, Bv = b (35) 
'error' otherwise 



U=\t: t(u, v) e {0, . . . , n}, v ) = 71 \ 



and D{v UtVt \\iiuv) = oo when the operation (35) returns an error. It should be noted that the cardinality of 
the set % is at most [n + l] 4 . 

We can apply the linear programming technique introduced in [9] to the construction of (35). By using the 
technique introduced in [9], the conditions Au = a and Bv = b can be replaced by linear inequalities. In the 
following, we focus on the condition t uv = t for a given t £ H. 
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Let it = (tii, • • • j u n ), v = (v\, . . . , v n ) and t = (i(0, 0), t(0, l),i(l, 0), t(l, 1)). First, we introduce the 
auxiliary variable s(ti, w) = (si(u, «),•■•, s n (u, v )) for (it, t>) € {0, l} 2 denned as 



Si(u,v) 



1, if Ui ~ u,Vi = v 
0, otherwise. 

Then the condition = t is equivalent to linear equalities 

n 

^2si(u,v) = t(u,v) 
»=i 

for all (it, u) e {0, l} 2 . 

Next, we show that the relation (36) can be replaced by linear inequalities. Let S(u, v) be defined as 



(36) 



S{u,v) = {(!,«,«)} U 



U {(0, «',«')} 

Then we obtain the convex hull of S(u,v) defined by the set of (s',u',v') that satisfies the following linear 
inequalities. 

s' > 

u' > if u = 

u' < 1 if u = 1 

i>' > if v = 

i>' < 1 if d = 1 

s' + [-1] V < 1 - u 

s' + [-l]V<l-« 

s' + [-l]V + [-l]V > 1-u-v, 

where constants it, v and variables s', it', and v are considered to be real numbers. For example, when (u, v) = 
(0, 0), we have 

5(0,0) = {(1,0,0), (0,0,1), (0,1,0), (0,1,1)} 

and linear inequalities 

s' > 
v! > 
d' > 
s' + t/ < 1 
s' + v' < 1 
s' + m' + d' > 1. 

Finally, we have 24n + 4 inequalities, which replace the condition t uv = t, defined as 

n 

y]sj(u,v) = t(u,v) 

i=l 
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> 


o 








U{ 


> 


o 


if u = 







Uj 


< 


1 


if u — 


1 




Vi 


> 





if v = 







Vi 


< 


1 


if v = 


1 


Si(u, v) + 


[-l] U Ui 


< 


1 


— u 




Si(u,v) + 




< 


1 


— V 




[-!]"«< + 




> 


1 


— u — V 


1 



where we take alH £ {1, 2, . . . , n} and (u, v) e {0, l} 2 . 

Remark 6: It should be noted here that linear programming frequently finds non-integral solutions for the 
operation (35) even if A and B are sparse matrices. Let us consider the polytope with vertexes Cj(a) x Cg(b). 
Then the section of the polytope cut by the hyperplane defined by t uv = t may have many fractional points. 
This is one reason why the linear programming finds non-integral solutions. In fact, the original operation 
introduced in [18] has the same problem. It remains a future challenge to find a good implementation of the 
operation (35). 

VII. Extension to Three or More Terminals 
In this section, we extend our results to three or more terminals. We use the following notations: 

/C={l,2,...,fc} 

U K = X Uj 

jeic 

u K = {(u jA ,u Jt2 ,... 7 u j . n )} jeK 
A K = {Aj} jeK 

and 

for each J C JC. 

A. Multiple Slepian-Wolf Source Code 

In this section, we consider the Slepian-Wolf coding for k correlated sources. 

Let X;c € X/c be correlated sources and [Lxk. be the joint probability distribution of Xjc- For each j e }C, 
we consider an ensemble (A.j , p^. ) of functions Aj : XJ 1 — > ImA, . We assume that function Aj is shared by 
the j-th encoder and the decoder. We define the j-th encoder as 

( x j) = AjXj 
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for each output Xj E X™ of the j-th source. Let gA K be defined as 

g Alc (a K ) = arg min D (v Xfz \\p.x K ) , 
where dj is the codeword of the j-th encoder and 

C A A a ic) = {uk ■ A-jUj = a d for all j E JC}. 

Then we define the decoder as 

ip' 1 (o K ) = 9a k («/c) ■ 

The construction of the above minimum-divergence decoder is described in Section VII-D. 
For each j E JC, the encoding rate Rj for the j-th encoder is given by 



Rj 



log |Im^4j| 



and the error probability is given by 

Errorx (A K ) = Hx K ({a* : ^ x ({<Pj {xj)} j€K ) + ^j) • 

We have the following theorem. 

Theorem 4: Assume that {Rj}- eK satisfies 

Y,Ri>H(Xj\Xjc) 

for all J C JC except the empty set, and an ensemble (A.j , p Aj ) has an (a Aj , /3a )-hash property for all j E JC. 
Then for any S > and all sufficiently large n there are functions (sparse matrices) A/c = {A,-} . £JC such that 

Errors (A K ) < S. 

B. Multiple Output Broadcast Channel Code 

In this section, we consider the fc-output broadcast channel coding. 

Let \iy k \x be the conditional distribution of a broadcast channel with input X E X and k outputs Y/e E 3^e- 
Let Uk, € Uk, be a multiple random variable and p,\j K be the joint probability distribution of [//£. Let 

F : — » X be a function that is allowed to be non-deterministic. The joint distribution hu^xy^ of (Uic,X, Yjc) 

is given by 

V-U K XY K {uK.,X,y K ) = flY K \x(yK\x)l*X\U K (x\UK.)l*U K (UK.), 

where Hx\u K represents the transition probability distribution of the function F. 

For each j E JC, we consider an ensemble (A.j,p A .) of functions Aj : UJ — > ImAj, where Uj is an alphabet 
of the source Uj. We also consider an ensemble (A!j,p A i ) of functions A'- : W> — » IraA^. Let rtij be the 
random variable corresponding to the j-th message, where the probability distribution p m are given by 



Pm J ( rn j) 



j^jrj if rrij E ImA'j 
0, if rrij ^ IraA'j 
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for each j £ JC. We assume that functions Aj, A'j and a vector a,j £ ImAj are shared by the encoder and the 
j-th decoder. For a multiple message £ ^-jeic IeoAj, we define the encoder <p : X ImAj — > Af" as 



?AK;A^(^/c,a/c) = arg 



mm 



-F 1 (m>c) = (F(uic,i), . . . , f(ujc,n)) 
where = {wj,i}jejt ^ or eacn * {!>•■•> n l an( ^ 



> . 



for all j e JC 

The construction of the above minimum-divergence decoder is described in Section VII-D. We define the j-th 
decoder (pj 1 : J 7 " — > Im^ as 

<M 3 (aj |y •) = arg max uj/ , y 3 (u, | y j ) 

UjGCAj- (aj) 

^ 7l (yj) = ^^(ojli/j)) 

for each output £ J 7 ] 1 - The error probability is given by 



Errors!* Afc, a K ) = 1 -^HYtfxivM™*:)) II [Pm,(mj)x {<Pj l {Vj) =m j ))]. 
We have the following theorem. 

Theorem 5: For each j £ JC, let Rj be the encoding rate for the j-th encoder. Assume that {(fj,Rj,£j)}j elc 
and e satisfy 

log |Iia4j| 



r, = 



log|Inv^| 



tj > H(Uj\Yj) 
E [Rj + rj] < H(Uj) e 



j&J 



(37) 

(38) 
(39) 
(40) 

(41) 



H(U K ,)-e<Y J \R 3 +r ] ]<H{U K ) 

jeic 

for all non-empty set J C JC and j £ JC. Furthermore, assume that an ensemble (A.j,p A .) (resp. (A!j,p A i )) 
has an (a/\ j , /3 Aj )-hash (resp. {a^, , (3 A , )-hash) property for all j £ JC. Then for any S > and all sufficiently 
large n there are functions (sparse matrices) and vectors (A/c, A' K , a/c) = { (Aj, A'-, aj) }j e)c such that 

Errory K | X {A K ,A' K ,a K ) < 5. 

It should be noted that there are {rj}jeic and e satisfying (37)-(41) if {Rj}j e /c satisfy 

E Rj < H(Uj) - E H(Uj\Yi) 



jej 



j&J 



E J (^) 



E H ^ - H ^ U ^ 

j&J 



(42) 
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for all non-empty set J C JC. The condition (42) is equivalent to the conditions 

Rj < IiUjiYj) for j g {1,2,3} 
J2i + R 2 < I(E/i; *i) + I(U 2 ;Y 2 ) - I(U i; U 2 ) 
R 1 + R 3 < /(f/i; Fi) + I(U 3 ; Y 3 ) - I(Vy, U 3 ) 
R 2 + R 3 < I(U 2 ; Y 2 ) + I(U 3 ; Y 3 ) - I(U 2 ; U 3 ) 
Rx + R 2 + R 3 < /(f/i; Yi) + I(U 2 ;Y 2 ) + I{U 3 ; Y 3 ) - J(t/ i; U 2 ) - I(U U U 2 ; U 3 ) 
specified in [8, Page 9-47], which considers the case of k — 3. 

C. Fundamental Lemmas for Theorems 4 and 5 

We can prove Theorems 4 and 5 in a similar way to Theorems 2 and 3 by using the following lemmas 
proved in Appendix E. In the following, we define 

\hnAj\ = J[ \lmAj\ (43) 

jej 

1, if J = 

\T\, \£J = K (44) 



\Tj\Jc 



max \Tuj\Ujc {ujc)\ , otherwise. 



Lemma 7: For each j <G JC, let Aj be a set of functions Aj : — > ImA, and p Aj be the probability 
distribution on Aj, where (Aj,p Aj ) satisfies (H3). Let the joint distribution Ph K a K be defined as 

Pk K * K (A]C, dfc) = JJ p Aj {Aj)p aj (ttj) 

for each {Aj, Oj}j e /f For each J C JC, let ol/\ j and /3 Aj be defined as 

«Aj = JJ OCAj (45) 

/3 Aj = H [0 Aj + 1] - 1. (46) 

Then 



PAK : [c? \ {i*/c}] n c Atc {a kUk) ± 0}) < £ ^nd^Ai±j + ^ 



(47) 



for all £ C [Wye]" an d U K € [Wjc]™, and 

p^. K ( { (^,a,) :rn^(a K ) = 0}) < « Ak - 1 + £ E^Mfe^ + l] 



(48) 



for all T C [W/c]"- Furthermore, if (a^, [3^.) satisfies (HI) and (H2) for all j e JC, then 

lim a hj (n) = 1 (49) 

n—yoo 

lim /3 Aj (n) = (50) 

n— >oc 

for every J C AC. 
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Lemma 8: If T is a subset of Tu K ,f> then 



for every non-empty set J' C /C. 



max |7^i W7C (uj-e)! < 2 "[ ff (^l^)+^" J =(TlT)] 



D. Construction of Minimum-Divergence Operation 

In this section, we introduce the construction of the minimum-divergence operation 

9A K (ijc) = arg min D {v UK \\fj. Ulc ) 

by assuming that Uj = {0, 1} is a finite field, and Aj is an l J \ j x n matrix on {0, 1} for all j E /C. It is a 
extension of the operation introduced in Section VI. 

Let t uic (b k ) E {0, . . . , n} be the number of occurrences of b k E {0, l} fe in u K E {0, l} kn . We have 

9a k (a/c) = arg min D [v u \\h Uk ) , 
ten 



where 



UK.t = < 



uic if 3ttjic s.t. t U)C = t, AjUj = aj for all j e /C 
'error' otherwise 



(51) 



|< : t (6 fc ) €{0,...,n},^t(6 fe ) = n j 



and D{v Utc t \\nu K ) = 00 when the operation (51) returns an error. It should be noted that the cardinality of 
the set % is at most [n + l] 2 *. 

We can apply the linear programming technique introduced in [9] to the construction of (51). By using the 
technique introduced in [9], the conditions AjUj = aj for all j e JC can be replaced by linear inequalities. In 
the following, we focus on the condition t UK = t for a given t e %. 

First, for each b k e {0, l} fc , we introduce the auxiliary variable (si (6 fc ) , . . . , s n (b k )) defined as 



U (b k ) 



1, if Uj :i = bj for all j E K, 
0, otherwise. 



(52) 



Then the condition t UfC = t is equivalent to the linear equality 



E s < (b k )=t(b k 



i=i 



for all b k E {0, l} fe . 

Next, we show that the relation (52) can be replaced by linear inequalities. Let S (b k ) C {0, l} k+1 be defined 

as 



S(b k ) _e {(l,b k )} U 



U {(M' fe )} 

b' k =ib k 



(53) 



We have the following lemma, which is proved in Appendix G. 

Lemma 9: The convex hull of S (6 fe ) is a polytope represented by the following inequalities: 



v > 
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Vj > for j e JC s.t. bj = 
vj < 1 for j e /C s.t. &j = 1 

u + [-Ipvj < 1 — 6j 

ie/c je/c 

where constants fe* and variables t>o, . . . , Vk are considered to be real numbers. Furthermore, there is no non- 
integral vertex of this poly tope. 

By using this lemma, we have [2[k + l]n + l]2 fe inequalities, which replace the condition t Utc = t, defined 

as 



i=i 

Sl (b k ) > 

u j,i > for j e JC s.t. bj = 

u j,i < 1 for j e /C s.t. bj = 1 

Si (b k ) + [-l] bi u jti <l-bj for all j e JC 

jeic jeic 
where we take all i € {1, 2, . . . , n} and b k e {0, l} fe . 

VIII. Proof of Theorems 

A. Proof of Theorem 1 

For a type t, let C t be defined as 

C t = {u G H" : t(u) = t} . 
We assume that £>a ({^4 : Am = 0}) depends on u only through the type t(u). For a given w e Ct, we define 

PA,t = PA ({A : Au = 0}) . 

We use the following lemma. 

Lemma 10 ([17, Lemma 9]): Let (a A ,f3 A ) be defined by (9) and (10). Then 

a A = \ImA\ max p A ,t (54) 

/3 A = 2 l Ct ^*' (55) 

where "H is a set of all types of length n except the type of the zero vector. 

Now we prove Theorem 1. It is enough to show (H3) because (HI), (H2) are satisfied from the assumption 
of the theorem. Since function A is linear, we have 

PA ({A : Au = Au'}) = PA ({A : A[u - u'} = 0}) 

= Ph,t{u-u') (56) 
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Then, for u ^ u' satisfying t(u — u') E Ha, we have 

Pa({A : Au = Au'}) = p A ,t(«-u') 

< maxp A ,t 
tew A 

= (57) 

where the last inequality comes from (54). Then we have the fact that Pa{{A : Au = Au'}) > aA/|Irru4| 
implies t(u — u') £ Ha- Finally, we have 

Pa ({A : Au = Au'}) < 

u'eu n \{u} u'eu n \{u} 

■TtsStr t( u -u')£H\H A 



< 



E E p*>* 

ten\H A u'eu n \{u} 

t{u—u')—t 

< E \ c *\p^ 

teu\H A 

= 13a, (58) 



where the equality comes from (55). 



B. Proof of Theorem 2 

Let (aj, y) be the output of correlated sources. We define 

•(x,y)tT XYn (SWi) 

• 3{x',y') ? (x,y) s.t. x' E C A (Ax), y' E C B (By), D(y x , y , \\fi XY ) < D(v xy \\fi XY ). (SW2) 

It should be noted that (SW2) includes the following three cases: 

• there is (x',y r ) satisfying x' ^ x, y' = y, x' E Ca(Ax), and D(v x > y > \\/j,xy) < D(v xy \\fj, X Y), 

• there is (x',y') satisfying x' = x, y' ^ y, y' E C B (By), and D(u x > y > \\hxy) < D{v xy \\n XY ), 

« there is (x',y') satisfying x' ^ x, y' ^ y, x' E Ca(Ax), y' E Cs(By), and D(v x > y > \\^xy) < 
D{v xv \\hxy)- 

Since a decoding error occurs when at least one of the conditions (SWI) and (SW2) is satisfied, the error 
probability is upper bounded by 

Error X y(A, B) < hxy{£i) + Vxy{£{ n £ 2 ), (59) 

where we define 

£ l = {(x,y):(SWt)}. 
First, we evaluate Eab [[J-xy(£i)]- From Lemma 22, we have 

Eab\hxy(Si)] < g (6°) 

for all sufficiently large n. 
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Next, we evaluate Eab [^xy(£i n E 2 )]- When (SW2) is satisfied but (SW1) is not, we have 

[Txy,-« \ {(*, y)}} n [C A (Ax) x C B (By)} ± 0. 
Applying Lemma 5 by letting Q = TxY,y, we have 

= ]T i2 XY (x,y)p AB ({(A,B):(SW2)}) 

(x,y)eTxY,-, 

( 

< ^XY(x,y)pAB 



(x.y)eTxY,-, 



|g|QA«B 

" |Im^||ImB| 



|7xy, 7 |aA«B 
" |Im^||ImB| 



V 



(A,B): 

[TxY.-t \ {(x,y)}} n [C A (Ax) x C B (By)] ^ 



max \Qx\y{y)\ 


a A [/3 B + 1] 


max |^|^(a;)| 




\lmA 


\lmB 





/3a + /?b + /?a/?b 



max |7i| Yi7 (y)| 



a A [/3 B + 1] 



M| 



+ 



max |7W 7 (a;) 



Q!b[^a + 1] 



|Im£ 



+ Pa + Pb + /?a/3b 



< 2"[ g ( x ^)+^y(^]g A a B 2"[ g ^l r )+^iy^l^q A [/? B + 1] 2^W)+^I^M]a B [/?A + i] 



|Im.4||ImB| |InU| 

+ /3 A + Pb + PaPb 

= 2 -n[Rx+RY-H(X,Y)- VX y(j)] a ^ aB + 2 -"[Rx-H(X\Y)-T, xl y( 1 \ 1 )] aA ^ B + X j 

+ 2 -n[R Y -H(Y\X)-r l y lx (jh)] aB ^ A + 1] + fa + + fafo 
< 6 - 

~ 4 



llmSI 



(61) 



for all sufficiently large n by taking an appropriate 7 > 0, where the third inequality comes from the fact 
that Q x C 7x, 7 . Qy C 7y, 7 , </*|;y(j/) C 7x|y, 7 (y), and ^^(a;) C 7V|x, 7 (a:), which are obtained from 
Lemma 20. The fourth inequality comes from Lemma 23 and the last inequality comes from (1 1)— (13) and 
conditions (HI), (H2) of ensembles (A.,p A ) and (B,p B ). 

Finally, from (59)-(61), for all S > and for all sufficiently large n there are A and B such that 

Error X y(A, B) < 5. 



Remark 7: We need the conditions x' e 73c, 7 and y' e 7y, 7 in the maximum-likelihood decoder (14) 
because £ = {(cc',y') : hxy{x' ,y') > fixY{x,y)} does not satisfy 

max 
V^Gy 

max \Qy\x{x)\ < 2 n[H{Y \ x)+ri y\x { ^ )] 
xeGx 

in general. It is unknown if we can remove these conditions. It seems that the same problem appears in [16, 
Eq. (20)]. 
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C. Proof of Theorem 3 

In the following, we assume that ensembles (A.,p A ), (A!,p A ,), (B,p B ), and (B',p B ,) have a strong hash 
property. Then, from Lemma 4, ensembles (A.,p A ) and (B,pg), defined by 

Au = {Au, A'u) 
Bv = (Bv,B'v) 

have an (a A , /3^)-hash property and an (ag, /3g)-hash property, respectively, where 

|Imi| = |Im.4||Im.4 / | 
\lm§\ = |ImB||ImS'| 

a A = a A a A > 
f3 x = f3 A + A , 

Otg = Ct B OLQi 

For f3 A and f3 B satisfying 

lim /3 A (n) = 

n— >oo 

lim /3 B (n) =0, 

n— !-oo 

let k. = {/s:(n)}^ 1 be a sequence satisfying 

lim k(u) = oo (62) 
lim K(n)/3 A (n) = (63) 
lim K(n)/3 B (n) = (64) 

n— >oo 

lim = o. (65) 

For example, we obtain such a k by letting 

if /3(n) = o(n-«) ,£ > 



-J= — , otherwise 



for every n, where /3(n) = m&x{(3 A (n), (3 B (n)}. If /3(n) is not o (n £), there is At' > such that /3(n)n^ > k' 
and 

log ac (n) _ lQ g^T 
n 2n 

- 2n 

£ log n - log k' 
= Yn (66) 

for all sufficiently large n. This implies that k satisfies (65). In the following, k denotes n{n). 
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From (20)-(25), and (65), we have the fact that there is 7 > such that 



£>%v(7) + ^ (67) 
n 

r Y >H(U\Y) + C u]y (2 1 \2 1 ) (68) 
r z >H(V\Z) + ( v \ z (2 1 \2 1 ) (69) 
r Y + R Y < H(U) - e - w\u(lh) (70) 
r z + R z <H(V)-e-rtu\v{l\i) (71) 
r Y +R Y + r z + R z <H(U,V)-r ]uv ('y)- ] ^ (72) 
+ R Y + r z + R z > H(U, V) - e (73) 

for all sufficiently large n. 
We have 

\Tuv„\ > 2"[ ff ( c/ ' v )-"»v(7)] 

= K\lmA\\lmA'\\ImB\\lmB'\ 

= K|Im^||ImB| (74) 

for all sufficiently large n, where the first inequality comes from Lemma 23, and the second inequality comes 
from (72). This implies that there is T C 7rjv, 7 such that 

ITI 

k < i, 1 - < 2k (75) 

|Im.4||IraB| 

for all sufficiently large n, where we construct such T by taking |T| elements from 7W, 7 in descending order 
of divergence. 

Now we prove the theorem. Let m and w be messages. Let u and v denotes the ^"-component (resp. 
V"-component) of qaa 1 bb 1 {a, m, b, w), that is, 

(u,v) = g A A'BB'{a,m,b,w). 

Let y and z be the channel outputs. We define 

• (u,v) e T c Tc/y, 7 (BCl) 

• (y, z) e Ty Z |c/yx, 7 (M, x) (BC2) 

• 5A(a|y)=w (BC3) 

• 5b(6|«) = «, (BC4) 
where we define x = f(u, v). Then the error probability (34) is upper bounded by 

ErTor YZlx (A 7 A',B,B',a,b) 

< PmwYz(Si) + p mwYZ (S2) + p mvi Yz(Si n S 2 n S3) + p mwYZ {S 1 n S 2 n 5|), (76) 
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where 



In the following, we define 



and 



Si = {(m,w,y,z):(BCi)}. 

a = (a, m) 
b= (b,w) 

C xs (a,b) = [C A (a)nC A ,(m)} x [C B (b)nC B >(w)] . 



First, we evaluate i?AA'BB'ab [PmwYz{Sf )]. From Lemma 6, we have 

-E/WBB'ab [PmwYz(Si)] 

= PAA'BB'abmw ({(-4, A', B, B' , a, b, m, w) : g A A<BB> (a, m, b, w) £ T}) 
< P A §ab ({(AB,a,b) :TnC xs (a,b) = 0}) 



llmBI 



< a^o;g — 1 + 



max \T u \v{v)\ 



in 



a x [0g + l] |W| 
+ 



max |T V | W (w)| 



«g[/ 3 A + 1 ] 



|T|n 



(77) 



+ 



|W||Imff| + ffg + fffrgg + 1] 

m 



From Lemma 20 and the fact that T C Txy,j, we have the fact that u e Tu implies u e Tu n and v <E Tv\u( u ) 
implies v e 7yi[/ i7 (ix). Furthermore, from Lemma 23, we have 



Then we have 



max|7^| V (^)| < max |7^|v, 7 («)l 

< 2 «[ff(^l^)+^|v(7l7)]. 



ImB| max |7^| V («)| o«[ff(c/|v)+r, U | V ( 7 | 7 )] 



(78) 



< 



/c|Iia4| 

2-"[ry+fly-H(i/|V)-jj M | V (7|7)] 



1 

< - 



(79) 



where the first inequality comes from (75), (78), and the second inequality comes from the fact that 

r Y + Ry = ry + Ry + r z + Rz - r z - Rz 

> H(U, V)-e-r z -R z 

> H(U, V) — e — H(V) + e + V u\v(lh) 

= H{U\V)+r lu]v { 1 \ 1 ) (80) 

which is obtained from (70) and (73). Similarly, we have 

|Ira4| max \T v \ u {u)\ 2 ~ n ^ z+Rz - H{v \ u ^-^\u(i\i)] 



\T\ 



< 
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< - (81) 

K 



Furthermore, we have 

' Im ^' < -. (82) 
\T\ K 

from (75). Then, from (62), (77), (79) (81), (82), and the properties (HI) and (H2) for ensembles (A, p^) and 
(B,pg), we have 

^AA'BB'ab [PmwY Z ($1)] < "| (83) 

for all <5 > and sufficiently large n. 

Next, we evaluate i?AA'BB'ab [PrnwYziS^)]- We have 

-^AA'BB'ab [PrnwYZ^)] = -E^BaE [^YZ\X ([Tyz\UVX,~/ (U,V,X)] C \X)] 

= ^ABab [»YZ\UVX ( [Tyz|^X, 7 (^, V, X)] " \U , V, X)] 

< 2-«[7-' )l uvA'yz] 

< s (84) 

for all (5 > and sufficiently large n, where we define X = f(U, V), the second equality comes from (18), 
and the first inequality comes from Lemma 22. 

Next, we evaluate i?AA'BB'ab [PmwYz(Si n S 2 n S3)] and i^AA'BB'ab [PmwYz(Si n S 2 n Sf)]. In the following, 
we assume (BC1), (BC2), and gA(a\y) 7^ u, where the last assumption is equivalent to gA(Au\y) 7^ u from 
(BC1). From (19) and Lemma 20, we have the fact that (it, v) e T implies (u, v, x) € Tuvx,-y- From Lemma 
20, we have (u,y) € Tvy.^ and it e Tu\Y,2~/(y)- Then there is it' € Ca(^Im) such that u' ^ u and 

Att/|y(«'|y) > Mc/|F(w|y) 

> 2 -™[H(C/|y)+C M |y(27|27)] ; ( 85 ) 

where the second inequality comes from Lemma 22. This implies that [Q{y) \ {u}] n Ca{Au) 7^ 0, where 

9(y) ee {«' : ^^(ti'ly) > 2-^(^l y )+C»i^^l 2 ^} . 

From Lemma 2, we have 

£ A [x(.9A(A«|y) ^ u)] < pa ({A : [0(y) \ {«}] n C A (Au) ? 0}) 

< 2 -n[r r -H (t /|y)-C M|y (27|27)] aA + ^ (g6) 

the last inequality comes from the definition of ry and the fact that 

\G(y)\ < 2 n[H{u \ Y)+ ^y {2 ^ 2 "< )] . 

Then we have 

-^AA'BB'ab [PmwYz(S\ fl S2 fl S3)] 



< ^AA'BB'abmw 



x(?AA'BB'(a,m,b,w) = (u,v)) ^ l*Y\x(y\x)x(9*{a\v) ^ «) 

(«,»)6T «e7V|i/vx,7('",'',a!) 
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< ^AA'BB'abn 



Y X(A« = a) x (A'w = m) x (Bv = b) x (B'v = w) 



Y »Y\x(y\x)x(g/i(a\y) ^ u) 

y&TY\uvx n (ii,v,x) 



= Y Y vv\x(y\x) 

(u,v)eT yeT Y \uvx,-,{u,v,x) 



Ea [x(flA(Au|y) ? u)E A , BB , abmw [ x (Au = a) x (A'u = m) x (Bv = b) x (B'v = w)] 
1 



7\ Y Y p Y \x{y\x)E A [ X (g A {Au\y) ^ u)] 



< 



|Im^||Im^'||IraB||Im#, 

2 -n[r Y -H(U\Y)-C ul y(2j\2j)] , o] l _ 

A PA J ^ |W||W||ImB||Im6'| 



< 2k 



2 -n[TY-H{U\Y)-C my (2 1 \2 1 )] ak + 



< \ (87) 

for all 5 > and sufficiently large n, where the second equality comes from Lemma 11 that appears in 
Appendix A, the fourth inequality comes from (75), and the last inequality comes from (63), (68), and the 
conditions (HI), (H2) of (A.,p A ). Similarly, we have 

^AA'BB'ab H S 2 H 5 4 C )] < 2k [ 2 -"[^- H ( y l Z )-^ |Z (27|27)] aB + 

< I (88) 

for all S > and sufficiently large n. 

Finally, from (76), (83), (84), (87), and (88), we have the fact that for all 5 > and sufficiently large n there 
are A e A, A 1 e A', B e B, B' eB',ae hnA, and b e ImB satisfying (34). ■ 

IX. Conclusion 

The constructions of the Slepian-Wolf source code and the broadcast channel code were presented. The 
proof of the theorems is based on the notion of a strong hash property for an ensemble of functions, where 
two lemmas called 'collision-resistance property' and 'saturation property' introduced [17] are extended from 
a single domain to multiple domains. Since an ensemble of sparse matrices has a strong hash property, we 
can construct codes by using sparse matrices and it is expected that we can use the efficient approximation 
algorithms for encoding/decoding. It should be noted that the capacity region for the general broadcast channel 
coding is unknown and we hope that our approach give us a hint for deriving the general capacity region. 

Appendix 

A. Basic Property of Ensemble 

We review the following lemma, which is proved in [17]. 

Lemma 11 ([17, Lemma 9]): Assume that random variables A and a are independent. Then, 

E a [x(Au = a)] = 
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for any A and u and 

^ Aa [x(An = a)] = ^ 

for any u. 

B. Proof of Lemma 1 

If an ensemble satisfies (H4), then we have 

E Pa ({A : Au = An'}) 



MET 

u'eT' 



= J2 Pa({A:A U = Au'})+Y, E P a({A:Au = Au'}) 

ueTnT' uer u 'eT'\{u} 

p A ({A:Au=Au'})<j^ l 

+ E E Pa({A:Au = Au'}) 

ueT u 'eT'\{u} 

p A ({A:Au=A-u'})> J1 ^ l 

uer u'er'\{u} 1 1 met 

pMA:Au=Au'})<^^ 

sirnri + eg^ + m* 

< \Tr\T\ + |7 S? A + mm{|71,|T'|}ft> (89) 
for any T and T' satisfying |T| < |T'|. ■ 

C. Proof of Lemma 4 
Let 

Pa,™,«' = Pa({-4 : Am = Au'}) 

Then we have 

E Pa({^ : ^-M) 
u'ew"\{u} 

= ^ ' PA.u.u'PA' ,u,u' 

u'eii n \{u} 

QA Q A' 

PA,u,»'PB,u,«'> |Im.A||ImB| 

= E PA,u,u'PA',u,u> + E PA,u,u'PA',«,«' 



u'eu"\{u} u'ew"\{u) 

,„ , °A°A' „ ,„ , , 

Pa,u.,u.' > |Imyl| PA,u,ii' < |ImyV| 



< E PA,u,u'PA',«,«' + E 

Pa, u.u'Pa' , u,u' 

u'eu n \{u} u'eu n \{u] 
PA,„,„'>irSar pa',„,^>|T^| 
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< E PA,-u,u' + ^ PA',u,i 

u'eu n \{u} u'eu n \{u} 



PA,u,u'> |I m .A| 

ft- 



Pa',«,«'> | Im V 



(90) 



where the first equality comes from the fact that A and A' are mutually independent and the second inequality 
comes from the fact that p A , u ,u' < 1 and pa', u ,w' < 1- Since (a^,/3^) satisfies (HI) and (H2), then we have 
the fact that (A., p^) has an (a^, /3^)-hash property. ■ 

D. Proof of Lemmas 5 and 6 
Let 



Pa,u, u > = Pa({A : Au = Au'}) 
PB,v,v> = Pb({A : Bv = Av'}). 



For (u',v') G T', we have 



^ PA,«,u'PB,i;,i;' = ^ PB,-u,-u' ^ PA,u,i 



(u,v)eT 



veTv 



Similarly, we have 



< 



E 

Pb,u,u' > TTS 



E 



"er U | V («) 



QA 
|Im.4| 



< 



max |7I/| V (v) 



"A 



< 



Ml 



max \T u \v(v)\ 

v£Tv 



E PB.v.V 

VeTv a 
PB,v,v' > ImS 



"A 



< 



Ml 



max \T u \v(v) 



E PB,d ; d' +PB,«,i; 

»ev"\{t.'} 

.PB,-u,-u' > |ImB| 



a A [/3 B + 1] 



|InU| 



(91) 



E PA,u,u>PB,v,v> < 

PA,u,u' > |l m ^4| 
Pb,i,,^'< llmBI 



max |"7v| W (m)| 



a B [/3 A + l] 



|ImB| 



(92) 
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for (it', v') e T'. We also have 



(u,«)6T 

Pa,h.,h.> > |Imi| 



PB,t,,„'>TT5^ 

< 



<|Tn{(u>')}| + £ PA,u,u'PB,v,v 



ue7l<\{u'} 

P/\,u,u' > Im.4| 



t>€7v\{t>'} 

PB r v,v' > |Im^3 



(u,o)6T 
u^u' 

. aft 
PA,ii,u' > |Im\A| 

Pb,»,»' > |I m B 



< 



|Tn + X! PA,u,u' + X! PB,t>,t>' + X! PA,«,u' J! PB,t),t>' 

ue«"\{u'} t!ev"\{t)'} «ew"\{«'} uev™\{V} 

PA,u,u' > |Im^4| PB,v.v' > |ImB Pa,u,u' > |Im^4| Pb.v,v' > |I m ^3 



PA,Ti,Ti'>J 

|T n {(«>')} I + /?a + /?b + /?a/?b 



< 

for (it', i/) e T'. Finally, we have 

Pab({(A,B):Au = Au , ,Bv = Bv'}) 

(u,v)eT 
(u'y)eT' 

= ^ ^ PA,u,u'PB,u,d' + ^ ^ PA,u,u'PB,v,v' 

(u'.v')eT' (u,t))er (u'.D'jer 1 («,«)er 

™,»,>,'- > [TmB] 



(93) 



"B.v .v' — |Imi3| 

+ E E 

(u',«')€T' (u,t>)er 

PA,w,Tt' > |I m ^4 
PB,v,v'^ |ImB 



P A ,li,-u'PB,v,v' + ^ ^ PA,u : u'PB,v,v' 

(u'.w')eT' (u,t))er 

Pa,™,-!*' > Im^4| 

PB,u,u' > |ImB 



x - |T|q;aQb , \ - 

- ^ |Im^||ImB| ^ 

(u',»')eT' 1 11 1 (u',v')eT' 

+ £ [\Tn{{u',v')}\+p A + p B + t3 A (i B 
(u'.v')eT' 



max \T u \v(v)\ 

veTv 



a A [/3 B + l] 



|Im.4| 



+ E 



max |7v|w(m)| 


a B [/3 A + l] 


\lmB 





IT' I 



+ 



rnax|7^|vCy)| 



a A [f3 B + l} |T' | 



max \T v \ u {u)\ 
ue%, 



a B [/3 A + l] 



|Im.4| 



llmBI 



(94) 



mrnoA«B 

- 1 1 |Im.4||ImS| 

+ \T'\ [Pa + p B + PaPb] , 

where the first inequality comes from (91)— (93). This implies that the joint ensemble (.4. x B,p AB ) satisfies 
(6) by letting 

a = «aqb 



ma? \%i\ v (v)\ 

veTv 


a A [/?B + l] 


max |T V |w(w)| 
ueTu 


a B [/3 A + l] 


\lmA 


i 


\lmB 





^ = |Wj + \lmB\ 

Then we have Lemmas 5 and 6 from Lemmas 2 and 3, respectively. 



+ /3 A + /3 B + /3 A /3 B . 
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E. Proof of Lemma 7 

It is easy to show (49) and (50) from the properties (HI) and (H2) of (a/\ d ,f3 A .) for all j e AC. In the 
following, we show (47) and (48). Let p u ,u'. be defined as 



P Uj ,u> = /'A ({Aj : = Aju'j}) . 



First, we have 



e n*s.«j 

uiceT jetc 
P^,„J<TI5%T forallj e J 

E II - E II/ " - 

„ , < 2J_ 



< 



max \T Uj \Ujc (ujc)\ 



n 



Q A 3 



e n p^.t 



< 



< 



max 17^1^.(^)1 

"J7 C £ 'Ujc 



max |7^| Mjc (u.7c)| 
|7>|j-<=| a Aj [p kjc + 1] 



n 



n 



llmA'l 



C*A 3 

|Im^- 



n 



E Pu^ur +Pu' j ,u' j 

u j&Jj\{ u 'j} 

P«,-,-u' > |Im^,-| 



n + 1] 



(95) 



for e T' and a non-empty set J C AC, where the second inequality comes from the property (H3) of 
(A/jPAj) for all j e /C and the fact that Pu^,u^ = 1> an d me l ast equality comes from (43) and (44). 
Next, we have 

E II /'« ,s E II /'» .s - II /'» >«j 

II E /'» « - 1 



n 



E *s. 



- 1 
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|70|/c|Q:A0 [/3a k + 1] 



- 1 



(96) 



\ImA \ 

for u' K € T', where we use the fact that PuJ.u^ = 1 in the second inequality, and the last inequality comes 
from (43)-(46). 
Finally, we have 

E Pa k {{A K : AjUj = AjUj for all j e /C}) 



= E E II/'" ■"■ 



< \TnT'\+ 



E 

P^.u'^TTS^n fora11 J e K 



EE E 



+ E E n^.s 

M !ceT' «teT\{%} ■ J ' e ' c 
P« i ,«'.>1T5^7T fora11 J ' e ' c 



<irnr'|+ X! 



E 

P«,X <TT^7T fora11 i eK 



n 



"A 3 



|j j 1 1 4 E E 



|7>|J- C | "Aj [/3ajc + l] 



E 



|7~0|d«A e [^Ak + 1] _ 1 
|Im^ | 



<irnri + CT^ + iri 

|Im.4jc| 



JCK 



(97) 



for all T, T' C x k 0=l Uj, where the first inequality comes from the fact that 

for all j e AC, the second inequality comes from (95) and (96). This implies that the joint ensemble (A-tcP^) 
satisfies (6) by letting 



a = «a k 



\p \7j\j-\ a *j [/3a + 1] _ 



J"C/C 



Then we have (47) and (48) from Lemmas 2 and 3, respectively, where we use the relation 



_ |gfl|d«A [fez + 1] _ 

c |W I 



for the proof of (47). 



E Proof of Lemma 8 

Since T is a subset of 7rj K , 7 , we have 



7^ c fu jc ,i 
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%(j\Ujc, 7 (ujc) c Tuj\ujc , 7 («j«) 
from Lemma 20. From Lemma 23, we have 

max |7^j| Mjc {ujc)\ < max |7^|i/ JC , 7 («j-=)| 

< 2 ™[- ff (C / jl^)+'/M J |Mjc(7l7)]^ 

■ 

G. Proof of Lemma 9 

Before proving the lemma, we prepare some definitions for a convex polytope (see [29]). Let V C R k be a 
convex polytope. Then the linear inequality < c is vaZ/c/ for V if this inequality is satisfied for all »eP. 
The face of the polytope "P is a set defined by 

Vn{v eR k : cv = c }, 

where cv < Co is valid for V . If the dimension of a face is 0, it is called a vertex. 

Lemma 12 ([29, Proposition 2.3(H)]): Every intersection of the faces of V is a face of V. 

Lemma 13 ([29, p. 52]): A point v e V cannot be expressed as a convex combination of V \ {v} if and 
only if it is a vertex of V . 

For k > 1, let V (6 fc ) C R fc+1 be a convex polytope defined by inequalities 



v a > (98) 

vj > for j e JC s.t. bj = (99) 

vj < 1 for j e /C s.t. &j = 1 (100) 

w + [-^Pvj <l-bj for all j e JC (101) 

oo + El-^^-E^ (102) 

ie/c jeic 



where constants b k G {0, l} fc and variables vo, . . . ,Vk are considered to be real numbers. 
Then we have the following lemmas. 

Lemma 14: Let v = (vo, Vi, . . . , Vk). Then, for all j G {0} U JC, inequalities 

-Vj < (103) 
Vj < 1 (104) 

are valid for V(b k ). 

Proof: First, we show (103) and (104) for j = 0. The inequality — vo < is valid because it is equivalent 
to (98). If &W = then we have 

Vo < v + Vi < 1 

from (99) and (101). On the other hand, if b 3 = 1 then we have 

vo < Vl < 1 
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from (100) and (101). Hence we have the fact that v < 1 is valid for V{b k ). 

Next, we show (103) and (104) for j e JC satisfying bj = 0. The inequality (103) is valid because it is 
equivalent to (99). The validity of (104) is obtained by 

Vj <v + vj < 1, 

which comes from (98) and (101). 

Finally, we show (103) and (104) for j e JC satisfying bj = 1. The inequality (104) is valid because it is 
equivalent to (100). The validity of (104) is obtained by 

Vj > vo > 0, 

which comes from (98) and (101). ■ 
Lemma 15: For all b k <E {0, l} fc , an integral point v in V{b k ) satisfies v e {0,l} fe+1 . 

Proof: From Lemma 14, the integral point v = (vo, v\, . . . , Vk) satisfies either Vj = or Vj = 1 for all 
j G {0} U K. ■ 
Lemma 16: For all b k <G {0, l} fc , (1,6 ) is a member of V(b ) and (0,M fc ) is a member for all 

€ {0, l} fc satisfying w fe 7^ 6 fc . 
Proof: The first statement (l,6 fc ) G V(b k ) is proved by showing that t> = (l,b k ) satisfies inequalities 
(98)-(102) for all b k e {0, 1}\ It is clear that v = (l,b k ) satisfies (98)-(100). Inequalities (101) and (102) 
come from the fact that v a = 1 and b k satisfies 

[-lpbj = -bj (105) 

for all j e JC. 

The second statement (0, u k ) e V(b k ) is proved by showing that v = (0, u k ) satisfies inequalities (98)-(102) 
for all b k e {0, l} k and u k e {0, l} fe satisfying u fc 7^ 6 fe . It is clear that u = (0, u k ) satisfies (98)-(100) for all 
u k e {0, l} fe . Since 

1 - 6,- , if 11, 7^ 60 
[-6j, if Uj = bj, 

we have 

+ [-l] b3 Uj < max{l - 6j, -bj} 

<l-bj (107) 
for all j e /C. This implies that v = (0,u k ) satisfies (101) for all j <G JC. From (106), we have 

£[-1]^= x: e ^ 

jeic jeic jeic 

Uj^bj Uj=bj 
J€/C j€/C 

>1-E 6 J' (108) 
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where the inequality comes from the fact that there is f e K such that uy ^ by because u k =^ b k . This implies 
that v = (0, u k ) satisfies (102). ■ 
Lemma 17: If b k e {0, l} fe , then (0, b k ) is not a member of V(b k ). If b k , u k e {0, l} fc satisfy u k ^ b k , then 
(1, u k ) is not a member of P(b k ). 

Proof: First, we show the first statement. From (105), we have 

o+£[-i] 6 ^ = -S> 

jeK jeK 

<1-J2by (109) 

jeK 

This implies that v = (0, b k ) is not a member of V(b k ) because it does not satisfy (102). 

Next, we show the second statement. Let us assume that b k ,u k £ {0, l} fc satisfy u k ^ b k . Then there is 
j' e fC such that uy ^ by. From (106), we have 

l + [-lp Uy =1 + 1 -by 

>l-by. (110) 

Therefore, we have the fact that v = (l,u ) is not a member of V(b k ) because it does not satisfy (101). ■ 

Lemma 18: The set of all integral vertexes of T'(b k ) is equal to S(b k ). 

Proof: From Lemmas 15-17, we have the fact that all integral points of "P(fe fc ) are members of S(b k ) and 
all members of S(b k ) are integral points of V(b k ). In the following, we show that all members of S(b k ) are 
vertexes of T'(b k ). 

We have 

k 

{v}=f]{v':v' j =v j } 

1=0 

for v ee (v , Vi,...,Vk) € S(b k ) C V(b k ). From Lemma 14 and the fact that v e {0, l} fe+1 , we have the fact 
that 

— v'j < Vj, for j e JC s.t. Vj = 
v j — v ji for j € JC s.t. Vj = 1 
are valid inequalities for V(b k ). This implies that 

T(b k )n{v> :v' J =v J } 
is a face of P(b k ). From Lemma 12, we have the fact that 

p| [V(b k ) n {«' : - «,•}] = V(b k ) n f) {«' : = «,•} 

= 7'(fe fe )n{t;} 

= M (in) 

is a face of V(b k ). Since the dimension of {v} is zero, we have the fact that v is a vertex of V(b k ). ■ 
Lemma 19: For all i; = (t>o, «i, . . . , Vk) € V{b k ) satisfying < vq < 1, w is not a vertex of V(b k ). 
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Proof: Let u k = (u^ ,. .., u^) be defined as 



vj — vob^ 



l-v 

Then v can be expressed as 

v = w o (l,& fe ) + [l-w o ](0,u fe ). (112) 

Since v ^ (0,u fe ) and v ^ (l,& fc ) for < vq < 1, we have the fact that u is not a vertex of V{b k ) from 
Lemma 13 by assuming that v, (l,b k ), and (0,u k ) are members of V(b k ). Then it is enough to show that 
(0,u k ) € "P^) by assuming that v e V(b k ) because we have (l,b k ) e V{b k ) from Lemma 16. 

In the following, wo is replaced by and Vj is replaced by in (98)-(102) when we state that (0, u k ) satisfies 
inequalities (98)-(102). We show that (0,u fc ) satisfies (98)-(102) by assuming v € V{b k ) and < v < 1. 
First, it is clear that (0,u k ) satisfies (98). Next, (0,u k ) satisfies (99) and (100) because 

U ) _ Vj - vob U) _ _w. 



for j e K, satisfying b^ = and 



u (j) = a -2i_ = > o 

1-vq 1 - w 



U U) = V J - v ° bU) = V J - v ° < ! 
1 - v 1 - v 



for j e K satisfying b^ = 1, where the inequalities come from the fact that w < 1 and Vj satisfies (99) and 
(100). Next, (0, u k ) satisfies (101) because 

1 - «0 

„ -wo-wo[-l] b<J> ^) 

1 - Vq 

_ [1 - feO")] [1 - W ] 

1 - «0 

= 1-&W (113) 

for all j e /C, where the inequality comes from the fact that w < 1 and Vj satisfies (101), the second equality 
comes from (105). Finally, we have the fact that (0,u fe ) satisfies (102) because 

v 



[-l] fcO) w,+w &« 



1 

> 



1 - wo 



= 1-^6^, (114) 

where the second equality comes from (105) and the inequality comes from (102) and the fact that < w < 1. 

■ 

Now we are in position to prove Lemma 9. From Lemma 18, it is enough to show that there is no non- 
integral vertex of V(b k ). Furthermore, from Lemma 19, it is enough to show that there is no non-integral vertex 
v = (w , wi, . . . , Vk) of V(b k ) by assuming w € {0, 1}. 
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Since V(b k ) C R fe+1 , the vertex of V(b k ) is determined by k + 1 equalities representing the face of V(b k ) 
described by 



VQ =0 

vj = for j e /C s.t. 6j = 
vj = 1 for j e /C s.t. 6j = 1 



«d + [-l] 6W) «j=l-6 W) forje/C 



(115) 
(116) 
(117) 
(118) 
(119) 



«*+E[-irW-E 6W) - 

ieK je/C 

Since w € {0, 1}, we have the fact that € {0, 1} if Vj is determined by one of the equalities (1 16)— (1 18). 

First, we consider the case where the vertex v — (u , v\, . . . , Vk) € "P(& fc ) does not satisfy (119). Then v 
should be determined by equalities (1 16)— (1 18) and we have the fact that Vj € {0,1} for all j e JC. This 
implies that v is an integral point of V{b k ). 

Next, we consider the case where the vertex v = (v$,v\, . . . ,Vk) € satisfies (119). Since k — 1 of k 

variables v\, . . . ,Vk should be determined by equalities (1 16)— (1 18), we have the fact that these variables are 
integers. Then, from (119) and the fact that v € {0, 1}, the remaining variable is also an integer. This implies 
that v is an integral point of V{b k ). 

Finally, from the above observations, we have the fact that all vertexes v — (vo, v\, . . . , Vk) € V(b k ) satisfying 
v a e {0, 1} are integral points. This implies that there is no non-integral vertex v = (vo,v\, . . . ,Vk) of V(b k ) 
that satisfies v a e {0, 1}. ■ 



H. Method of Types 

We use the following lemmas for a set of typical sequences. 

Lemma 20 ([28, Theorem 2.5][17, Lemma 23]): If v e 7V, 7 and u e Tu\v,i' ( v )> tnen ( u > v ) € Tuv,t+-y'- 
If (u,v) e 7t7v,7> tnen u € 7c7, 7 and u e Tu\V.-y( v )- 

Lemma 21 ([28, Theorem 2.7][17, Lemma 25]): Let < 7 < 1/8. Then, 

1 



- lo 8 



for all u € 7t7, 7 , and 



log- 



H(U) 



H(U\V) 



< 0/(7) 



< Cw|v(7'l7) 



n nu\ v (u\v) 

for e 7V,7 and it e 7t/|y. 7 '(f), where C,u{l) and Cw|v(7'l7) are defined in (2) and (3), respectively. 
Lemma 22 ([28, Theorem 2.8][17, Lemma 26]): For any 7 > 0, and v e V™, 

M C /|y([Ta|y,7^)] c |t;)<2-"[T- A "v] ; 

where \u and A^y are defined in (1). 

Lemma 23 ([28, Theorem 2.9] [17, Lemma 27]): For any 7 > 0, 7' > 0, and v e 7y, 7 , 

1 



n 



log |Tt7, 7 | - ^(17) 



< %/(7) 
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±log\T ulv ,Y(v)\-H(U\V) <^| V ( 7 '| 7 ), 
where r}u{l) and rj U \ v {^' \"f) are denned in (4) and (5), respectively. 
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